{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "52dc9b17-1182-44b3-bebf-ae2f508675d3",
   "metadata": {},
   "source": [
    "# Handling Large Multi-Modal Payloads in AgentCore Runtime\n",
    "\n",
    "## Overview\n",
    "\n",
    "This tutorial demonstrates how Amazon Bedrock AgentCore Runtime handles large payloads up to 100MB, including multi-modal content such as Excel files and images. AgentCore Runtime is designed to process rich media content and large datasets seamlessly.\n",
    "\n",
    "### Tutorial Details\n",
    "\n",
    "|Information| Details|\n",
    "|:--------------------|:---------------------------------------------------------------------------------|\n",
    "| Tutorial type       | Large Payload & Multi-Modal Processing|\n",
    "| Agent type          | Single         |\n",
    "| Agentic Framework   | Strands Agents |\n",
    "| LLM model           | Anthropic Claude Sonnet 4 |\n",
    "| Tutorial components | Large File Processing, Image Analysis, Excel Data Processing |\n",
    "| Tutorial vertical   | Data Analysis & Multi-Modal AI                                                   |\n",
    "| Example complexity  | Intermediate                                                                     |\n",
    "| SDK used            | Amazon BedrockAgentCore Python SDK|\n",
    "\n",
    "### Key Features\n",
    "\n",
    "* **Large Payload Support**: Process files up to 100MB in size\n",
    "* **Multi-Modal Processing**: Handle Excel files, images, and text simultaneously\n",
    "* **Data Analysis**: Extract insights from structured data and visual content\n",
    "* **Base64 Encoding**: Secure transmission of binary data through JSON payloads"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a676f58ecf52b42",
   "metadata": {},
   "source": [
    "## Prerequisites\n",
    "\n",
    "* Python 3.10+\n",
    "* AWS credentials configured\n",
    "* Docker running\n",
    "* Sample Excel file and image for testing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "initial_id",
   "metadata": {},
   "outputs": [],
   "source": [
    "#!uv add -r requirements.txt --active\n",
    "#!uv add openpyxl --active"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "932110e6-fca6-47b6-b7c5-c4714a866a80",
   "metadata": {},
   "source": [
    "## Create Sample Data Files\n",
    "\n",
    "Let's create sample Excel and image files to demonstrate large payload handling:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "create-sample-data",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from PIL import Image, ImageDraw, ImageFont\n",
    "import os\n",
    "\n",
    "# Create a large Excel file with sample sales data\n",
    "np.random.seed(42)\n",
    "data = {\n",
    "    'Date': pd.date_range('2023-01-01', periods=1000, freq='h'),\n",
    "    'Product': np.random.choice(['Widget A', 'Widget B', 'Widget C', 'Gadget X', 'Gadget Y'], 1000),\n",
    "    'Sales': np.random.randint(1, 1000, 1000),\n",
    "    'Revenue': np.random.uniform(10.0, 5000.0, 1000),\n",
    "    'Region': np.random.choice(['North', 'South', 'East', 'West'], 1000),\n",
    "    'Customer_ID': np.random.randint(1000, 9999, 1000)\n",
    "}\n",
    "\n",
    "df = pd.DataFrame(data)\n",
    "df.to_excel('large_sales_data.xlsx', index=False)\n",
    "\n",
    "# Create a sample chart image\n",
    "img = Image.new('RGB', (600, 500), color='white')\n",
    "draw = ImageDraw.Draw(img)\n",
    "\n",
    "# Draw a simple bar chart\n",
    "products = ['Widget A', 'Widget B', 'Widget C', 'Gadget X', 'Gadget Y']\n",
    "values = [250, 180, 320, 150, 280]\n",
    "colors = ['#FF6B6B', '#4ECDC4', '#45B7D1', '#96CEB4', '#FFEAA7']\n",
    "\n",
    "max_value = max(values)\n",
    "bar_width = 120\n",
    "start_x = 100\n",
    "\n",
    "for i, (product, value, color) in enumerate(zip(products, values, colors)):\n",
    "    x = start_x + i * (bar_width + 20)\n",
    "    height = int((value / max_value) * 400)\n",
    "    y = 500 - height\n",
    "    \n",
    "    # Draw bar\n",
    "    draw.rectangle([x, y, x + bar_width, 500], fill=color)\n",
    "    \n",
    "    # Add labels (simplified without font)\n",
    "    draw.text((x + 10, 510), product[:8], fill='black')\n",
    "    draw.text((x + 10, y - 20), str(value), fill='black')\n",
    "\n",
    "draw.text((300, 50), 'Sales Performance by Product', fill='black')\n",
    "img.save('sales_chart.png')\n",
    "\n",
    "# Check file sizes\n",
    "excel_size = os.path.getsize('large_sales_data.xlsx') / (1024 * 1024)  # MB\n",
    "image_size = os.path.getsize('sales_chart.png') / (1024 * 1024)  # MB\n",
    "\n",
    "print(f\"Excel file size: {excel_size:.2f} MB\")\n",
    "print(f\"Image file size: {image_size:.2f} MB\")\n",
    "print(f\"Total payload size: {excel_size + image_size:.2f} MB\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "agent-code",
   "metadata": {},
   "source": [
    "## Create Multi-Modal Agent\n",
    "\n",
    "Let's create an agent that can process both Excel files and images from large payloads:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3b845b32-a03e-45c2-a2f0-2afba8069f47",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile multimodal_data_agent.py\n",
    "from strands import Agent, tool\n",
    "from strands.models import BedrockModel\n",
    "import pandas as pd\n",
    "import base64\n",
    "import io\n",
    "import json\n",
    "from bedrock_agentcore.runtime import BedrockAgentCoreApp\n",
    "\n",
    "app = BedrockAgentCoreApp()\n",
    "\n",
    "# Initialize the model and agent\n",
    "model_id = \"us.anthropic.claude-sonnet-4-20250514-v1:0\"\n",
    "model = BedrockModel(\n",
    "    model_id=model_id,\n",
    "    max_tokens=16000\n",
    ")\n",
    "\n",
    "agent = Agent(\n",
    "    model=model,\n",
    "    system_prompt=\"\"\"\n",
    "    You are a data analysis assistant that can process large Excel files and images.\n",
    "    When given multi-modal data, analyze both the structured data and visual content,\n",
    "    then provide comprehensive insights combining both data sources.\n",
    "    \"\"\"\n",
    ")\n",
    "\n",
    "@app.entrypoint\n",
    "def multimodal_data_processor(payload, context):\n",
    "    \"\"\"\n",
    "    Process large multi-modal payloads containing Excel data and images.\n",
    "    \n",
    "    Args:\n",
    "        payload: Contains prompt, excel_data (base64), image_data (base64)\n",
    "        context: Runtime context information\n",
    "    \n",
    "    Returns:\n",
    "        str: Analysis results from both data sources\n",
    "    \"\"\"\n",
    "    prompt = payload.get(\"prompt\", \"Analyze the provided data.\")\n",
    "    excel_data = payload.get(\"excel_data\", \"\")\n",
    "    image_data = payload.get(\"image_data\", \"\")\n",
    "    \n",
    "    print(f\"=== Large Payload Processing ===\")\n",
    "    print(f\"Session ID: {context.session_id}\")\n",
    "    \n",
    "    if excel_data:\n",
    "        print(f\"Excel data size: {len(excel_data) / 1024 / 1024:.2f} MB\")\n",
    "    if image_data:\n",
    "        print(f\"Image data size: {len(image_data) / 1024 / 1024:.2f} MB\")\n",
    "    print(f\"Excel data {excel_data}\")\n",
    "    print(f\"Image data {image_data}\")\n",
    "    print(f\"=== Processing Started ===\")\n",
    "    # Decode base64 to bytes\n",
    "    excel_bytes = base64.b64decode(excel_data)\n",
    "    # Decode base64 to bytes\n",
    "    image_bytes = base64.b64decode(image_data)\n",
    "    \n",
    "    # Enhanced prompt with data context\n",
    "    enhanced_prompt = f\"\"\"{prompt}\n",
    "    Please analyze both data sources and provide insights.\n",
    "    \"\"\"\n",
    "    \n",
    "    response = agent(\n",
    "        [{\n",
    "            \"document\": {\n",
    "                \"format\": \"xlsx\",\n",
    "                \"name\": \"excel_data\",\n",
    "                \"source\": {\n",
    "                    \"bytes\": excel_bytes\n",
    "                }\n",
    "            }\n",
    "        },\n",
    "        {\n",
    "            \"image\": {\n",
    "                \"format\": \"png\",\n",
    "                \"source\": {\n",
    "                    \"bytes\": image_bytes\n",
    "                }\n",
    "            }\n",
    "        },\n",
    "        {\n",
    "            \"text\": enhanced_prompt\n",
    "        }]\n",
    "    )\n",
    "    return response.message['content'][0]['text']\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    app.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "setup-infrastructure",
   "metadata": {},
   "source": [
    "## Setup Infrastructure and Deploy Agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "54dd2fdf-985c-4a70-8b87-071783a209de",
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "import os\n",
    "\n",
    "# Get the current notebook's directory\n",
    "current_dir = os.path.dirname(os.path.abspath('__file__' if '__file__' in globals() else '.'))\n",
    "\n",
    "# Navigate up to the utils.py location\n",
    "utils_dir = os.path.join(current_dir, '..')\n",
    "utils_dir = os.path.join(utils_dir, '..')\n",
    "utils_dir = os.path.abspath(utils_dir)\n",
    "\n",
    "# Add to sys.path\n",
    "sys.path.insert(0, utils_dir)\n",
    "\n",
    "from utils import create_agentcore_role\n",
    "\n",
    "agent_name = \"multimodal_data_agent\"\n",
    "agentcore_iam_role = create_agentcore_role(agent_name=agent_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2e79eba2-ca59-463f-9ebf-56e362d7ae66",
   "metadata": {},
   "outputs": [],
   "source": [
    "from bedrock_agentcore_starter_toolkit import Runtime\n",
    "from boto3.session import Session\n",
    "\n",
    "boto_session = Session()\n",
    "region = boto_session.region_name\n",
    "\n",
    "agentcore_runtime = Runtime()\n",
    "\n",
    "response = agentcore_runtime.configure(\n",
    "    entrypoint=\"multimodal_data_agent.py\",\n",
    "    execution_role=agentcore_iam_role['Role']['Arn'],\n",
    "    auto_create_ecr=True,\n",
    "    requirements_file=\"requirements.txt\",\n",
    "    region=region,\n",
    "    agent_name=agent_name\n",
    ")\n",
    "\n",
    "launch_result = agentcore_runtime.launch()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "wait-for-deployment",
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "\n",
    "status_response = agentcore_runtime.status()\n",
    "status = status_response.endpoint['status']\n",
    "end_status = ['READY', 'CREATE_FAILED', 'DELETE_FAILED', 'UPDATE_FAILED']\n",
    "\n",
    "while status not in end_status:\n",
    "    time.sleep(10)\n",
    "    status_response = agentcore_runtime.status()\n",
    "    status = status_response.endpoint['status']\n",
    "    print(f\"Deployment status: {status}\")\n",
    "\n",
    "print(f\"Final status: {status}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "test-large-payloads",
   "metadata": {},
   "source": [
    "## Test Large Multi-Modal Payloads\n",
    "\n",
    "Now let's test the agent with large payloads containing both Excel data and images:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3d909e42-e1a0-407f-84c2-3d16cc889cd3",
   "metadata": {},
   "outputs": [],
   "source": [
    "import base64\n",
    "import uuid\n",
    "import json\n",
    "from IPython.display import Markdown, display\n",
    "\n",
    "# Encode files to base64\n",
    "with open('large_sales_data.xlsx', 'rb') as f:\n",
    "    excel_base64 = base64.b64encode(f.read()).decode('utf-8')\n",
    "\n",
    "with open('sales_chart.png', 'rb') as f:\n",
    "    image_base64 = base64.b64encode(f.read()).decode('utf-8')\n",
    "\n",
    "# Create large payload\n",
    "large_payload = {\n",
    "    \"prompt\": \"Analyze the sales data from the Excel file and correlate it with the chart image. Provide insights on sales performance and trends.\",\n",
    "    \"excel_data\": excel_base64,\n",
    "    \"image_data\": image_base64\n",
    "}\n",
    "\n",
    "session_id = str(uuid.uuid4())\n",
    "print(f\"📊 Processing large multi-modal payload...\")\n",
    "print(f\"📋 Session ID: {session_id}\")\n",
    "print(f\"📄 Excel size: {len(excel_base64) / 1024 / 1024:.2f} MB\")\n",
    "print(f\"🖼️ Image size: {len(image_base64) / 1024 / 1024:.2f} MB\")\n",
    "print(f\"📦 Total payload: {len(json.dumps(large_payload)) / 1024 / 1024:.2f} MB\\n\")\n",
    "\n",
    "# Invoke agent with large payload\n",
    "invoke_response = agentcore_runtime.invoke(\n",
    "    large_payload,\n",
    "    session_id=session_id\n",
    ")\n",
    "final_response = \"\"\n",
    "for r in invoke_response['response']:\n",
    "    final_response += r.decode(\"utf-8\")\n",
    "response_data = json.loads(final_response)\n",
    "display(Markdown(response_data))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cleanup",
   "metadata": {},
   "source": [
    "## Cleanup Resources"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cleanup-resources",
   "metadata": {},
   "outputs": [],
   "source": [
    "import boto3\n",
    "\n",
    "# Clean up AWS resources\n",
    "agentcore_control_client = boto3.client('bedrock-agentcore-control', region_name=region)\n",
    "ecr_client = boto3.client('ecr', region_name=region)\n",
    "iam_client = boto3.client('iam')\n",
    "\n",
    "# Delete AgentCore Runtime\n",
    "runtime_delete_response = agentcore_control_client.delete_agent_runtime(\n",
    "    agentRuntimeId=launch_result.agent_id\n",
    ")\n",
    "\n",
    "# Delete ECR repository\n",
    "ecr_client.delete_repository(\n",
    "    repositoryName=launch_result.ecr_uri.split('/')[1],\n",
    "    force=True\n",
    ")\n",
    "\n",
    "# Delete IAM role\n",
    "policies = iam_client.list_role_policies(\n",
    "    RoleName=agentcore_iam_role['Role']['RoleName'],\n",
    "    MaxItems=100\n",
    ")\n",
    "\n",
    "for policy_name in policies['PolicyNames']:\n",
    "    iam_client.delete_role_policy(\n",
    "        RoleName=agentcore_iam_role['Role']['RoleName'],\n",
    "        PolicyName=policy_name\n",
    "    )\n",
    "\n",
    "iam_client.delete_role(RoleName=agentcore_iam_role['Role']['RoleName'])\n",
    "\n",
    "# Clean up local files\n",
    "os.remove('large_sales_data.xlsx')\n",
    "os.remove('sales_chart.png')\n",
    "\n",
    "print(\"✅ Cleanup completed!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "conclusion",
   "metadata": {},
   "source": [
    "# Congratulations!\n",
    "\n",
    "You have successfully demonstrated handling large multi-modal payloads with Amazon Bedrock AgentCore Runtime!\n",
    "\n",
    "## What you've learned:\n",
    "\n",
    "### Large Payload Processing\n",
    "* **100MB Support**: AgentCore Runtime can handle payloads up to 100MB\n",
    "* **Base64 Encoding**: Secure transmission of binary data through JSON payloads\n",
    "* **Efficient Processing**: Runtime optimized for large data processing\n",
    "\n",
    "### Multi-Modal Capabilities\n",
    "* **Excel Analysis**: Processing structured data from spreadsheets\n",
    "* **Image Processing**: Analyzing visual content and charts\n",
    "* **Combined Analysis**: Correlating insights from multiple data types\n",
    "\n",
    "### Key Benefits\n",
    "* **Rich Data Processing**: Handle complex, multi-format datasets\n",
    "* **Scalable Architecture**: Runtime designed for large workloads\n",
    "* **Tool Integration**: Custom tools for specialized data processing\n",
    "* **Enterprise Ready**: Secure handling of sensitive business data\n",
    "\n",
    "This demonstrates AgentCore Runtime's capability to handle enterprise-scale data processing tasks with multiple data modalities, making it ideal for complex business intelligence and data analysis applications."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
